Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLOUDP-211733 - Pipeline and build process refactoring #1427

Merged
merged 32 commits into from
Nov 28, 2023

Conversation

Julien-Ben
Copy link
Collaborator

@Julien-Ben Julien-Ben commented Nov 16, 2023

Refactoring of pipeline.py

Overview

  • The pipeline.py file was refactored to reduce duplication and simplify the build process.
  • Initially, there was a lot of duplication with different functions for building each image type (e.g., build_agent_image_ubi, build_version_post_start_hook_image). These shared 90% of their logic and configuration.
  • The refactoring process introduced a unified function, build_and_push_image, to handle the building and pushing of images.

Key Changes

  1. Unified Build Function:

    • A single function build_and_push_image replaced the multiple image-building functions.
    • This function centralizes the image-building process, reducing redundancy.
  2. New Argument Building Function:

    • Introduced build_image_args to define the build arguments passed to Sonar.
    • This function starts with a default argument dictionary, then edits the fields for handling special cases as necessary.
  3. Simplified Argument Handling:

    • Replaced specific arguments passed to sonar like e2e_image, operator_image, etc., with a generic image argument.
    • Adjustments in Sonar inventory files to align with the new generic image parameter.
    • Removed the agent-ubuntu variant in inventory.yaml.
  4. Unified Variant Naming:

    • Standardized --image-name flag values in VALID_IMAGES_NAMES to be consistent with values in ~/.community-operator-dev/config.json.
    • For example, operator in pipeline corresponds to operator_image in dev config, and version_upgrade_hook corresponds to version_upgrade_hook_image. Dev images have the _image_dev prefix. No need for any explicit mapping anymore.
    • Harmonized naming in release.json too.
    • Eliminated -ubi prefixes for operator and agent images and removed the logic for building Ubuntu-based images.
  5. Adjustments in E2E Tests and Makefile:

    • Updated to reflect new image names and ensure correct pipeline argument passing.
  6. Argument Parsing Changes:

    • Modified the behavior of the --release flag: now, it's either added to signify a release or omitted otherwise (instead of --release true or --release false).
    • Introduced a new architecture flag to choose the build architecture, defaulting to multi-architecture if not specified.
  7. Documentation Updates:

    • Updated documentation to reflect changes in the local development configuration.
  8. Additional Improvements:

    • Implemented a new function for running CLI commands with improved error handling, remove a lot of redundancy.
    • Enhanced manifest pushing logic: the --amend argument is now added only for the correct architectures to prevent errors from non-existent image names.

@Julien-Ben Julien-Ben changed the title Pipeline and build process refactoring CLOUDP-211733 - Pipeline and build process refactoring Nov 16, 2023
@Julien-Ben Julien-Ben marked this pull request as ready for review November 20, 2023 13:37
Copy link
Contributor

@slaskawi slaskawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added two questions, otherwise LGTM

helm-charts Outdated Show resolved Hide resolved
.action_templates/jobs/setup.yaml Show resolved Hide resolved
Copy link
Member

@mircea-cosbuc mircea-cosbuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with the question about the helm-chart submodule bump.

@Julien-Ben Julien-Ben merged commit f3ff90f into master Nov 28, 2023
39 of 40 checks passed
@Julien-Ben Julien-Ben deleted the refactor_pipeline branch November 28, 2023 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants